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Problem Statement 

Current theories of learning have emphasized the value of dialogue for student engagement and 
achievement (Cazden, 2001; Bruffee, 1992). Research has also shown that the nature of classroom discourse 
depends greatly on the teacher (Anderson, Rourke, Garrison, & Archer, W., 2001). These issues are relatively 
well understood in face-to-face classrooms. However, the advent of online learning has raised more questions 
about student interaction and the role of teachers in such contexts. We need to develop a better understanding of 
how teachers can provide effective online mentoring and scaffolding to facilitate student engagement with each 
other and with their subject matter (Bonk, 2003). 

Perceptions of online teachers’ roles in distance education remain quite varied and controversial 
(Lobel, Neubauer, & Swedburg, 2002). Although online instruction literature increasingly emphasizes the 
importance of moderation and leadership (Anderson, et ah, 2001; Feenberg, 1989), it remains unclear how 
online moderating are related to student engagement and interaction. The purpose of this research is to develop 
a better understanding of the relationship between moderator behavior and student engagement in synchronous 
computer conferencing learning environments. 

Theoretical Perspectives 

Current interest in collaborative learning — both in face-to-face and computer- mediated classrooms — is 
grounded in socio -cultural and socio -constructive theories (Duffy & Cunningham, 1996; Vygotsky, 1934, 

1978). Learning is seen as a process of negotiating community membership through various social interactions 
(Wenger, 1998) with peers, experts, and teachers (Kaye, 1992). Proponents of computer conferencing have 
often argued that such social-constructivist perspectives may be particularly amenable to this new medium 
(Bonk & King, 1998; Bruce & Levin, 1997). They argue that computer conferencing may help students 
maximize both their own and peers’ learning through the use of collaborative activities and discussions. 

The teacher plays an important role in online discussions. While individual learning can occur through 
independent or self-directed study, it is only through active intervention of a teacher or moderator that 
collaborative computer conferencing becomes a useful instructional and learning resource (Garrison, Anderson, 
& Archer, 2001). Though the literature recommends (e.g. Garrison & Anderson, 2003; Salmon, 2000) extensive 
online moderating and guidelines, few experimental studies evaluate, much less certify, moderating processes or 
validate the optimal level or scope of online moderating. 

Research Questions 

As we have argued above, the relationship between moderating level and student engagement is 
complicated. Thus the major task of this study is to investigate the relationship between teacher moderating 
levels and student engagement. Taking a mixed research method approach, both quantitative and qualitative 
questions are asked: 

1. Quantitatively, how are teacher moderating levels associated with each of the three 

student engagement variables? Is the effect of moderating levels on one student engagement 

variable higher than on another student engagement variable? 

2. Qualitatively, what does the process of the collaborative meaning construction look 
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like? What is the transactional nature of the relationship between teacher moderating 
levels and student engagement? 

As the first step of a large scale study, our answers to the research questions are based on a 
preliminary analysis of a small portion of the large data set. Additional analyses are underway and will 
be presented. 



Research Design 

The current research applies a mixed method approach — a combination of qualitative and quantitative 
approaches. Its quantitative character is evident in the process of converting communication content into 
discrete units and calculating the frequency of occurrence of each unit. It is quantitative also in that it extends 
the descriptive results of content analysis to inferential hypothesis testing (Borg & Gall, 1989; Rourke, 
Anderson, Gairison, Archer, 2001) which intends to certify the relationship between the predictor variable of 
moderating levels and outcome variables of student engagement. 

Data Collection 

The prime data source for this study will be the automatically archived conference transcripts from an 
online three-credit course offered at a Canadian University. This course on interpersonal communication is 
delivered through a real-time, interactive text, image, and animation messaging system and it is one of the first 
synchronous technologies that offer a visual representation of participant interaction. This course is unique also 
in the respect that all activities and interactions happen in real-time, i.e. in synchronous mode (Lobel, et al., 
2002 ). 

Though the transcripts form the main data for the study we plan to collect other data to triangulate the 
results (Patton, 2002). These additional data sources include course syllabus; course readings; classroom 
activity agenda developed by the teaching team and delivered to each teaching staff two days before class once 
a week; class preparation — one hour online meeting of the teaching staff immediately before class, and course 
assignments. These data will help better understand the context of each conference. 

Data Analysis 

The predicator variable of the study is moderating level. By adapting and combining Xin’s (2002) 
rubric for measuring online moderating with Anderson et al.’s (2001) teaching presence model, this researcher 
has created a five-level rubric to measure the moderating level. In this model, the minimal level of moderating 
(level 1) includes when the moderator; opens discussion, establishes the computer conferencing agenda, and 
observes conference norms. At the high end of moderating (Level 5), the moderator strongly weaves and 
summarizes participants’ ideas in addition to performing the previous moderating levels or functions. Though 
termed as “levels,” the scale embraces both the quality and quantity nature of moderating. 

The outcome variables of the study are student engagement and its sub-constructs. Student engagement 
is measured through three indicators (sub-constructs): behavioral engagement, social-emotional engagement, 
and intellectual engagement. While computer log data provides behavioral engagement information, emotional 
engagement is assessed through emotional expression and group cohesion attributed to closeness, warmth, 
affiliation, attraction, and openness (Rourke et al, 1999). Interactivity and higher-order thinking are considered 
key indicators of intellectual engagement in this inquiry. In terms of computer conference interactivity, 
declarative, reactive, and interactive messages are coded (Hara et al., 2000; Henri, 1992; Rafaeli & Sudweek, 
1996; Sarlin et al., 2003). In terms of higher-order thinking, messages of problem initiation, problem 
exploration, and idea integration are coded. 

Given that the synchronous conferencing messages are relatively short, content analyses focus on 
individual message units as the unit of analysis. A message unit is considered a posted message that is 
automatically numbered by the system. Inter-rater reliability (Krippendorf, 1980) is determined using Cohen's 
kappa. 



Preliminary Findings and Discussion 

With the purpose of revealing relationships between teacher moderating behaviors and student 
engagement which may later lead to the articulation of a model or framework for online teacher moderation, we 
have conducted preliminary analyses of the data here. To date, we have looked at the relationship between 
number of teacher postings and student attending and participating, two indicators of student behavioral 
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engagement. Second, we compared the outcome variable (student engagement) of two different groups — one 
with an overall high moderating level (Level 5) and the other with a somewhat lower moderating level (Level 
2 ). 

To look at the relationship between predictor variable teacher posting and outcome variables attending 
and participation, a correlation analysis was performed. The Pearson correlation between teacher posting and 
student attending as well as student participation were significant, respectively. The comparison of a portion of 
the outcome variable (student engagement) of two different groups — one with an overall high moderating level 
(Level 5) and the other with a somewhat lower moderating level (Level 2) provided interesting findings. 
Findings show that in these two groups behavioral engagement and intellectual engagement are about the same, 
whereas emotional engagement is at least two to three times higher in the group with low moderating level. 
Further analysis of one indicator of intellectual engagement — higher order thinking — shows that there are 
minimal differences in the frequency of problem initiation. However, the level of exploration is higher in the 
group with a high moderating level compared to the group with a low moderating level. In contrast, the level of 
idea integration is higher in the group with low moderating level compared to the group with high moderating 
level. 

One interesting finding that emerged was the striking difference in the number of creative ideas offered 
by students in these two groups. The group with a high moderating level only produced three main ideas for 
their project, whereas the other group with a low moderating level produced up to seven different ideas for their 
project. Even though idea generation was not considered intellectual engagement in prior research, we think 
these differences are worth exploring in the other online classes. We are interested in the number of unique 
ideas or solutions produced by group members since it is a sign of creativity and divergent thinking. 

Our pre liminary analysis of a small sample of the data indicates some interesting trends. As we 
forecasted, there appear to be a difference in student engagement levels when group discussions are moderated 
at different levels. However, higher levels of moderation might not be associated with higher levels of student 
engagement. Consequently, we believe that there is a need to discover how to effectively moderate student 
behavioral, emotional, and intellectual engagement. Higher moderating levels might be more conducive for 
more student exploration, but may interfere with idea integration, emotional expression, and creative ideas. 

Significance of the Study 

This study explore s student engagement in relation to online moderating in synchronous computer 
conferencing. Eventually, research in this area can extend to online training programs and curricula. The results 
of the study could allow researchers and practitioners develop better protocols for moderating online 
discussions. Such knowledge is essential if online learning (particularly synchronous conferencing) is to achieve 
its full potential. 
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